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ABSTRACT 



A statewide performance-based accreditation system began 
using a performance index in 1995-96 to indicate where school districts fall 
in the distribution of accreditation levels. The purpose of this study was to 
determine whether there was a significant gain in the performance index in 
the second year. The performance index was based on the percentage of 
performance variables met by a district and was designed to assist district 
personnel in assessing improvement toward higher accreditation levels. 
District personnel were provided with a table indicating the total number of 
variables involved in the accreditation system and the performance index 
assigned for meeting each number. The performance index, used in conjunction 
with their district scores, indicated to personnel the number of variables on 
which the district needed improvement and the specific areas requiring 
improvement. The performance index was assigned in March 1996, and the second 
index was assigned in March 1997 for 1996-97. An analysis was done to 
determine if the gain in the 1996-97 performance index was significant over 
the 1995-96 index for the approximately 150 districts in the state. The 
difference was found to be significant (p<.01). A flow chart for the 
accreditation system is provided in an appendix. (Contains two figures, three 
tables, and seven references.) (Author/SLD) 
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Abstract 

The purpose of this study was to determine whether there was a significant gain in the 
performance index in its second year. 

A statewide performance-based accreditation system began using a performance index in 1995-96 
to indicate where school districts fall in the distribution of accreditation levels. The performance 
index was based on the percentage of performance variables met by a district and was designed to 
assist district personnel in assessing improvement toward higher accreditation levels. District 
personnel were provided with a table indicating the total number of variables involved in the 
accreditation system and the performance index assigned for meeting each number. The 
performance index, used in conjunction with their district scores, indicated to the personnel the 
number of variables on which the district needed improvement and the specific areas requiring 
improvement. 

The performance index was first assigned in March 1996, and the second index was assigned in 
March 1997 for 1996-97. An analysis was done to determine if the gain in the 1996-97 
performance index was significant over the 1995-96 index for the approximately 150 districts in 
the state. The difference was found to be significant (p < .01). 
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In 1994-95 the Mississippi Commission on School Accreditation and the State Department of 
Education developed a performance index to be used in the statewide performance-based 
accreditation system. The performance index was designed to help district personnel judge the 
effectiveness of their district relative to the overall accreditation model. The data “help monitor 
and assess performance. Just as goals are an essential element of success, data are an essential 
piece of working towards goals” (Schmoker, 1996, p. 29). The establishment of the performance 
index was discussed by Jarrell, Hebbler, Troiani, and Dyson (1995). The performance index was 
officially assigned for the first time in 1995-96 and the second time in 1996-97. The purpose of 
this study was to determine whether there was a significant gain in the performance index in its 
second year. 

Background 

As Ladd (1996) pointed out, “present efforts to reform elementary and secondary education in the 
United States are focusing heavily on the outcomes of the educational system” (p. 1). The 
Mississippi accreditation system is a performance-based or outcomes-based system. The 
statewide performance-based accreditation system began using a performance index in 1995-96 to 
indicate where school districts fall in the distribution of accreditation levels. In this system 
districts receive an accreditation level of 1 (Probation), 2 (Warned), 3 (Successful), 4 (Advanced), 
or 5 (Excellent) based on compliance with both performance and process standards (see Appendix 
A). Levels 1 through 3 (Phase 1) are based on a group of standards set at a minimum of 
adequacy. Those districts meeting Level 3 standards are then evaluated against the Level 5 
standards with higher criteria in Phase 2 of the system. Because the five accreditation levels do 
not allow much discrimination among the districts, the performance index was designed to 
provide more distiction within the accreditation levels. As Elmore, Abelmann, and Fuhrman 
(1996) indicated, the Mississippi performance index was created “to account for differences in 
performance among school districts assigned to the same level” (p. 75). The performance index 
was aligned with the accreditation levels assigned to districts by the performance-based 
accreditation system. A district with an accreditation level of 1 due to performance would have a 
performance index between 1.0 and 1.9, depending on the percentage of performance variables 
met. A district with an accreditation level of 2 due to performance would have a performance 
index between 2.0 and 2.9. The performance index ranged from 1.0 to 5.0. The performance 
index was based on the percentage of performance variables met by a district and was designed to 
assist district personnel in assessing improvement toward higher accreditation levels. District 
personnel are provided each year with a table indicating the total number of standards involved in 
the accreditation system and the performance index assigned for meeting each possible number of 
standards. Table 1 shows the performance index for 1995-96, and Table 2 shows the 
performance index for 1996-97. The performance index, used in conjunction with their district 
scores on the district’s Performance Report (Table 3), indicated to the personnel the number of 
variables on which the district needed improvement and the specific areas requiring improvement. 
The performance index was intended as a strategy “aimed at empowering . . . local educators” 
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(Sagor, 1996, p. 2), while the Performance Report provides data “for correcting unwanted 
deviations from desired outcomes” (Duffy, 1996, p. 185). 

In the Mississippi accreditation model, a district meets a performance standard by having a district 
average score that meets or exceeds the annual criterion for that standard. For example, the grade 
4 reading score from the norm-referenced test is one of the performance standards. In 1996-97, 
the annual criterion was an NCE of 37.0 in Phase 1 and 47.2 in Phase 2. The district in the 
example in Table 3 had an NCE of 41.8; that district met the Phase 1 minimum but not the Phase 
2 minimum. The district’s performance index was a 3.0, indicating that the district was at the 
lower end of the Level 3 districts. A performance index of 3.0 tells the district personnel that 
there are many Phase 2 performance standards that the district did not meet. In this particular 
case, there are also a few Phase 1 performance standards that the district did not meet. If the 
district wants to raise its accreditation level, it must improve the scores of its students on each of 
the standards that it failed to meet. 

By using the performance index and the Performance Report, personnel can see at a glance how 
close the district is to earning a higher accreditation level, how many standards for which they 
must improve the student’s scores, how close they are to attaining each standard, and where the 
district needs to concentrate its curriculum and instruction efforts. 

One problem with the performance index exists because of the changing number of standards each 
year. The performance-based accreditation system is constantly evolving due to the different 
statewide tests available for use. In 1995-96 there were 35 standards in Phase 1 and 38 in Phase 
2. In 1996-97 a new subject area test in Algebra 1 was piloted to replace the former test, as a 
result the Algebra 1 scores were not included in the accreditation system. This meant only 34 
standards in Phase 1 and 37 in Phase 2 for that year. At the same time that the Algebra 1 test was 
being piloted, a new U.S. History test was piloted, therefore in 1997-98 there will be 36 standards 
in Phase 1 and 39 in Phase 2. Each change in the number of standards requires a corresponding 
change in the performance index that is based on the percent of standards met. 

Another problem is lack of sensitivity to the amount by which a district missed a standard. The 
performance index is based on whether the district met or did not meet the standards; there is no 
way for the index to measure how far the district was from meeting each standard that it failed to 
meet. For instance, the district in Table 3 failed to meet the Phase 1 Performance Assessment in 
Mathematics at grade 7 by 0.3 scale score points. The students were very close to performing at 
the level set by the Commission on School Accreditation as adequate. Another district might 
have missed the same standard by 75 scale score points, however both districts failed to meet the 
standard. If that were the only standard that both districts failed to meet, their performance 
indices would be the same. But the first district would not need to work as hard as the second to 
meet that standard the next year. (See Figure 1.) 



2 




5 



Performance Index 




| District A gj District B 



Figure 1 



If two districts are compared simply on their reading scores at grade 4-9, one can see another 
example of the lack of sensitivity in the performance index. In this example (Figure 2), District A 
has met all but one of the reading standards. On the one standard that it failed to meet, grade 6 
reading, the score is a significant distance from the annual criterion, and the district would have to 
work diligently with those students to improve their scores. If that were the only Phase 1 
standard that the district failed to meet, its performance index would be at least 3.0 depending on 
how many Phase 2 standards it met. District B failed to meet each of the reading standards, but 
the scores in each case are barely below the Phase 1 criteria. This district would not have to work 
as hard to improve the scores enough to meet the standards, however its performance index 
would be a 2.6. 
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Figure 2 



Despite limitations due to the varying number of standards and the lack of sensitivity in the 
performance index, district personnel see it as a means of monitoring their status in the 
performance-based accreditation system. Administration and faculty work constantly to improve 
the educational experience for their students, and according to Kochan (1996), “performance on 
standardized tests remains the predominant criterion for measuring school effectiveness” (p.l). 
The performance index gives school personnel a gauge to see whether they are succeeding. A 
district might remain at an accreditation level of 2 for several years, but if the personnel can see 
that their performance index is rising each year, they can make the argument that they are 
improving. Many times proof of improvement is necessary for satisfying the public and the state 
department. 
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Table 1 

1995-96 Performance Index 







ipji Number of Variables 


f4: : Number MetBP 


: Percent 


l^pindeiGlil 




5 


38 


38 


100.0% 


5.0 






38 


37 


97.4% 


4.9 






38 


36 


94.7% 


4.7 




4 


38 


35 


92.1% 


4.5 






38 


34 


89.5% 


4.3 






38 


33 


86.8% 


4.1 


p 

H 




38 


31-32 


81.6-84.2% 


3.9 


A 




38 


29-30 


76.3-78.9% 


3.8 


s 

E 




38 


27-28 


71.1-73.7% 


3.7 






38 


25-26 


65.8-68.4% 


3.6 


L 




38 


23-24 


60.5-63.2% 


3.5 




3 


38 


21-22 


55.3-57.9% 


3.4 






38 


19-20 


50.0-52.6% 


3.3 






38 


17-18 


44.7-47.4% 


3.2 






38 


15-16 


39.5-42.1% 


3.1 






38 


<=14 


<=36.8% 


3.0 






35 


31 


88.6% 


2.9 






35 


30 


85.7% 


2.7 






35 


29 


82.9% 


2.6 




2 


35 


28 


80.0% 


2.5 






35 


27 


77.1% 


2.3 






35 


26 


74.3% 


2.1 


P 




35 


25 


71.4% 


2.0 


H 




35 


24 


68.6% 


1.9 


A 

S 




35 


23 


65.7% 


1.8 


E 




35 


22 


62.9% 


1.7 


1 




35 


21 


60.0% 


1.6 






35 


20 


57.1% 


1.5 




1 


35 


19 


54.3% 


1.4 






35 


18 


51.4% 


1.3 






35 


17 


48.6% 


1.2 






35 


16 


45.7% 


1.1 






35 


<=15 


< =42.9% 


1.0 
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Table 2 

1996-97 Performance Index 







$ Number of Variables 


Number Met: : 


!; ’ Percent Met •• '^!§; 


Index 




5 


37 


37 


100.0% 


5.0 








36 


97.3% 


4.9 








35 


94.6% 


4.7 




4 


37 


34 


91.9% 


4.5 








33 


89.2% 


4.3 








32 


86.5% 


4.1 


p 

H 

A 






30-31 


81.1-83.8% 


3.9 






28-29 


75.7-78.4% 


3.8 


S 

E 






26-27 


70.3-73.0% 


3.7 








24-25 


64.9-67.6% 


3.6 


2 




37 


22-23 


59.5 -62.2% 


3.5 




3 


20-21 


54.1-56.8% 


3.4 








18-19 


48.6-51.4% 


3.3 








16-17 


43.2-45.9% 


3.2 








14-15 


37.8-40.5% 


3.1 








< = 13 


<=35.1% 


3.0 








30 


88.2% 


2.9 








29 


85.3% 


2.7 








28 


82.4% 


2.6 




2 


34 


27 


79.4% 


2.5 








26 


76.5% 


2.3 








25 


73.5% 


2.1 


P 






24 


70.6% 


2.0 


H 

A 






23 


67.6% 


1.9 


A 

S 






22 


64.7% 


1.8 


E 






21 


61.8% 


1.7 


1 






20 


58.8% 


1.6 




1 


34 


19 


55.9% 


1.5 




18 


52.9% 


1.4 








17 


50.0% 


1.3 








16 


47.1% 


1.2 








15 


44.1% 


1.1 








<=14 


<=41.1% 


1.0 
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Table 3 



PERFORMANCE BASED ACCREDITATION 
1996-97 PERFORMANCE REPORT 



District: OOOO County School 



Date: 03/14/97 







District 


Level 3 




Level 5 




Performance Standard 


Value 


Minimum 


Met 


Minimum 


Met 


01 


FLE Reading 


265.6 


257.5 


Yes 


274.4 


No 


02 


FLE Math 


254.8 


257.5 


No 


273.2 


No 


03 


FLE Written Comm. 


262.2 


255.0 


Yes 


269.2 


No 


04 


ITBS 4 Reading 


41.8 


37.0 


Yes 


47.2 


No 


05 


ITBS 4 Language 


50.1 


38.8 


Yes 


48.2 


Yes 


06 


ITBS 4 Mathematics 


44.7 


40.0 


Yes 


48.7 


No 


07 


ITBS 5 Reading 


42.8 


37.0 


Yes 


46.2 


No 


08 


ITBS 5 Language 


54.5 


38.5 


Yes 


48.9 


Yes 


09 


ITBS 5 Mathematics 


44.7 


38.6 


Yes 


49.4 


No 


10 


ITBS 6 Reading 


47.2 


39.3 


Yes 


49.5 


No 


11 


ITBS 6 Language 


57.4 


39.9 


Yes 


47.4 


Yes 


12 


ITBS 6 Mathematics 


49.0 


40.0 


Yes 


47.6 


Yes 


13 


ITBS 7 Reading 


40.9 


37.0 


Yes 


48.2 


No 


14 


ITBS 7 Language 


45.8 


39.7 


Yes 


50.0 


No 


15 


ITBS 7 Mathematics 


45.0 


37.9 


Yes 


48.0 


No 


16 


ITBS 8 Reading 


41.4 


40.0 


Yes 


51.7 


No 


17 


ITBS 8 Language 


42.2 


40.0 


Yes 


50.1 


No 


18 


ITBS 8 Mathematics 


43.0 


37.3 


Yes 


47.9 


No 


19 


TAP 9 Reading 


39.5 


37.0 


Yes 


47.0 


No 


20 


TAP 9 Language 


45.4 


40.0 


Yes 


49.8 


No 


21 


TAP 9 Mathematics 


41.4 


37.0 


Yes 


45.9 


No 


22 


PA 4 Int Lang Arts 


441.6 


400.0 


Yes 


480.6 


No 


23 


PA 4 Mathematics 


415.7 


400.0 


Yes 


463.6 


No 


24 


PA 5 Int Lang Arts 


468.5 


400.0 


Yes 


478.6 


No 


25 


PA 5 Mathematics 


413.1 


400.0 


Yes 


453.0 


No 


26 


PA 6 Int Lang Arts 


478.0 


400.0 


Yes 


478.8 


No 


27 


PA 6 Mathematics 


471.1 


400.0 


Yes 


463.1 


Yes 


28 


PA 7 Int Lang Arts 


436.1 


400.0 


Yes 


468.4 


No 


29 


PA 7 Mathematics 


399.7 


400.0 


No 


454.9 


No 


30 


PA 8 Int Lang Arts 


436.5 


400.0 


Yes 


472.6 


No 


31 


PA 8 Mathematics 


406.6 


400.0 


Yes 


441.1 


No 


32 


PA 9 Int Lang Arts 


443.1 


400.0 


Yes 


467.9 


No 


33 


PA 9 Mathematics 


401.7 


400.0 


Yes 


441.5 


No 


34 


% in 1st Quarter 


Met 3/7, 2/7 


30% on 5/7 


No 


25% on 5/7 


No 


35 


ACT Composite Score 


17.1 


N/A 


N/A 


20.2 


No 


36 


IHL (% ACT Core) 


43.1% 


N/A 


N/A 


35% 


Yes 


37 


Graduation Rate 


70.7% 


N/A 


N/A 


75% 


No 



Percentage of Level 3 Performance Standards Met: (31 / 34) 91 .2% 

Percentage of Level 5 Performance Standards Met: ( 6 / 37) 1 6.2% 



Your District's 1996-97 Performance Index §j 3*0- j 

Your 1996-97 accreditation level will depend on the information on this report plus compliance with 
the accreditation process standards. See Bulletin 171 (12th Edition, Revised, pg. 19) for complete 
information on the performance standards and interpretation of the performance index. On variable #34 the 
district value indicates the number of tests on which the district met the Level 3 and the Level 5 minimums. 
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Analysis 

The performance index was first assigned in March 1996, and the second index was assigned 
in March 1997 for the 1996-97 school year. An analysis was done to determine if the gain in 
the 1996-97 performance index over the 1995-96 index for the 153 districts in the state was 
significant. The average performance index in 1995-96 was 3.1, while the average 
performance index in 1996-97 was 3.2. The differences showed 32 districts had a decrease 
ranging from 0.1 to 1.5 points, 53 districts remained at the same performance index, and 68 
districts showed an increase ranging from 0. 1 to 2.5 points. A t-test on the differences 
between the two indices was significant (p < .01). 

Conclusions 

Despite the statistical significance, care must be taken in interpreting the change. As noted 
earlier, there are weaknesses in the calculation of the performance index due to the varying 
number of performance standards from year to year. The performance index is, however, 
more helpful than the accreditation level in distinguishing among the districts’ performance. 
During years when the number of standards in the accreditation system is stable, the 
performance index will provide a better gauge than in those years when there are standards 
added or dropped. 
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